Reinforcement Learning from Human Preferences-trained GPT-3 like model
Reinforcement Learning from Human Preferences-trained GPT-3 like model
CarperAI, a new research lab within the EleutherAI research collective, aims to democratize the "LLMs" "instruction-tuning" of large language models, the same way Stable Diffusion democratized image generation.
Last week, CarperAI released trlX, the first public implementation of the technique that can be used to train models with billions of parameters, to widespread acclaim. Today, they're going a step further and announcing a broad coalition aimed at training and publicly releasing instruction-tuned models with EleutherAI and Multi, experts in training large language models, and Scale, Humanloop, and HuggingFace, experts in labelling and human annotation.
@humanloop: Today we're excited to announce that we're partnering with @CarperAI of Stability on bringing the first RLHF-trained GPT-3 like model to the open source community. This will be huge. Let us explain
@humanloop: RLHF – Reinforcement Learning from Human Preferences. Models are fine tuned using RL from human feedback. They become more helpful, less harmful and they show a huge leap in performance. An RLHF model was preferred over a 100x larger base GPT-3 model. https://pbs.twimg.com/media/FfeTNJCUcAEVuo5.jpg
@humanloop: These models show far greater ability to take instruction, which has massively increased their usability. We think RLHF-tuned models will ultimately be applied to every domain and task, and these systems will unlock incredible amounts of value in the real world.